This invention relates to a display control apparatus and method for controlling a stereoscopic display device which allows a user to observe a stereoscopic image utilizing parallax between the right and left eyes. Further, the invention relates to a display control apparatus and method for controlling a stereoscopic display device which allows a user to observe a stereoscopic image by reproducing image points in space.
A display system available in the prior art makes it possible to present a display that switches between or mixes a two-dimensional display and a three-dimensional display. Typical means for displaying a stereoscopic image are binocular parallax and volume-scanning. The method using binocular parallax allows the observer to observe a stereoscopic image by utilizing a slight difference in the images perceived by the right and left eyes, namely binocular parallax. Volume-scanning involves presenting a time-division display of a number of cross-sectional images of an object while changing the display plane in the depth direction in observed space by moving the image forming system, and causing the image cross sections to emerge in superimposed form in observed space by utilizing the visual persistence of the human eye.
Further, known computer systems come equipped with a graphical user interface as a user interface intended to improve operability. A graphical user interface and a stereoscopic display mechanism will now be described in greater detail.
(1) Graphical user interface
In order to operate a computer and enter data, use is made of input devices such as a keyboard, tablet mouse or trackball, etc. In particular, for objects that are easy to recognize visually, such as windows, icons and pull-down menus, a tablet and mouse often are used together with a graphical user interface (hereinafter a "GUI") to enter coordinate information and path information into a computer while the monitor is being viewed. These input devices and the monitor display make it possible to perform operations intuitively and facilitate the operation of the computer.
FIG. 48 is a hierarchical diagram of a typical computer system in which a GUI is used in the operating system. Shown in FIG. 48 is application software (referred to simply as an "application" below) 500 utilized by a user, and a visual shell 501, which is the environment that allows the user to actually operate the computer interactively and visually. The application 500 constructs the visual shell of the GUI environment and controls external peripherals by utilizing a GUI component application interface (API) 502, a GUI component library/server 503, a paint API 504 for painting images on a display 506, a paint library/server 505, another API 507 for using other peripherals 509, another library/server 508, and a device driver 510 for controlling each of the devices.
The progress that has been made in computer software and hardware has been remarkable and developments in the area of display devices has been no exception, with progress having been made in providing high-quality colonization, larger display surfaces and higher definition. A concurrent tendency is to provide displays that allow stereoscopic viewing, as well as the pursuit of greater quantities of information and realism in displays. Several of such schemes have been proposed and put into practice.
(2) Stereoscopic display mechanism using binocular parallax
A stereoscopic image display system having a parallax barrier (referred to as a "parallax barrier system") is well known as a system for presenting a stereoscopic display on a display device.
A parallax barrier system has been disclosed in "Theory of Parallax Barriers" by S. H. Kaplan, J.SMPTE, Vol. 59, No. 7, pp. 11.about.21 (1952). In the parallax barrier system, a striped image having left and right images in an alternating array is displayed from among a plurality of parallax images obtained from a plurality of viewpoints. Parallax images corresponding to the eyes are observed by respective ones of the eyes via a slit (referred to as a "parallax barrier") having a prescribed aperture provided at a position spaced a prescribed distance away from the striped image. The corresponding parallax images can thus be observed by the respective eyes, thereby making stereoscopic viewing possible.
A stereoscopic display apparatus disclosed in the specifications of Japanese Patent Application Laid-Open Nos. 3-119889 and 5-122733 is designed to improve the compatibility with a two-dimensional (single-viewpoint image) image display device. Specifically, a parallax barrier is generated electronically by a transmissive-type liquid crystal display element, and control is performed electronically to change the shape and position of the barrier stripes.
FIG. 49 is a basic structural view of the stereoscopic image display device disclosed in Japanese Patent Application Laid-Open No. 3-119889. As shown in FIG. 49, a transmissive-type liquid crystal display device 101 which presents an image display and an electronic parallax barrier 103 comprising a transmissive-type liquid crystal display element are disposed on either side of a spacer 102 having a thickness d. Parallax images captured from two or a number of directions are displayed as vertical stripe images on the transmissive-type liquid crystal display device 101. By designating XY addresses using a controller such as a microcomputer 104, a parallax barrier pattern can be formed at any position on the barrier surface of the electronic parallax barrier 103, thus making possible stereoscopic vision in accordance with the principle of the parallax barrier scheme described above.
FIG. 50 is a structural view showing the display section of a stereoscopic image display device comprising the liquid crystal panel display 101 and electronic parallax barrier 103 disclosed in Japanese Patent Application Laid-Open No. 3-119889. As shown in FIG. 50, each of two liquid crystal layers 115, 125 is embraced by two polarizers 111, 118 and 121, 128, respectively. When a two-dimensional display is presented in this device, the display using the electronic parallax barrier is turned off to produce a colorless, transparent state over the entire image display area. By performing control in this manner, compatibility with a two-dimensional display is realized, unlike the case with the stereoscopic image display device using the conventional parallax barrier.
FIGS. 51A.about.51C are diagrams illustrating differences in parallax barrier patterns formed on an electronic parallax barrier owing to a difference in number of viewpoints. When stripe images formed from parallax images for two viewpoints are observed, the width A of opaque portions which block light and the width B of light transmitting portions of the parallax barrier are advantageously the same, as shown in FIG. 51A. On the other hand, as the number of viewpoints increases to three viewpoints and six viewpoints, the aperture efficiency of the electronic parallax barrier declines, as illustrated in FIGS. 51B and 51C.
Further, as shown in FIGS. 52A and 52B, Japanese Patent Application Laid-Open No. 5-122733 discloses an example in which a pattern of barrier stripes is generated only in part of the area of an electronic parallax barrier 103 comprising a transmissive-type liquid crystal display element, wherein a three-dimensional image and a two-dimensional image are capable of being displayed in mixed form on the same screen.
In addition to the parallax barrier scheme, a lenticular scheme is well known as means for displaying a stereoscopic image using the binocular parallax of the right and left eyes. In a lenticular system, a lenticular sheet comprising a number of semicylindrical lenses is provided in front of the display to spatially separate the image that enters the right and left eyes so that the observer is allowed to see a stereoscopic image.
(3) Three-dimensionally visible image display mechanism using volume-scanning
An example of an apparatus which uses volume-scanning to present a stereoscopic display will be described with reference to FIG. 53. As illustrated in FIG. 53, a laser beam of reduced diameter is caused to scan in two dimensions using a light deflecting scanner 301, whereby an image is displayed on an oscillating screen 302. The screen 302 is moved back and forth in the depth direction at high speed and the image of the cross section of an object at the position of the screen 301 is painted by the laser beam in sync with the back-and-forth movement of the screen. Repeating this operation at high speed causes a three-dimensional image to appear owing to the effect of persistence of vision.
In accordance with this method, a three-dimensionally visible image having a shape the same as that of the object is formed in observation space. Even if the observer moves to the left or right, therefore, the observer will always see an object the same as that of the original object that would be seen from that position. This makes possible natural viewing at all times.
However, a system which implements a GUI through use of the conventional display that relies solely upon a display in two dimensions does not have means for allowing the system itself to judge whether an object such as a window or icon manipulated by the user is an object capable of being displayed three-dimensionally or an object displayed two-dimensionally. Consequently, the following problems arise when use is made of a system capable of presenting a three-dimensional display:
(1) Consider a situation in which a plurality of windows or icons or the like are being displayed by an apparatus which presents a three-dimensional display and a two-dimensional display. When any object is activated to shift the current of the application, whether there is to be a changeover from the two-dimensional display to the three-dimensional display, a changeover from the three-dimensional display to the two-dimensional display or no changeover whatsoever cannot be determined on the side of the host computer. This means that the user must make the designation or effect the changeover each time. Similarly, with an apparatus configured to mix a two-dimensional display and a three-dimensional display, it is required that the user specify, object by object, whether the object is a two-dimensional display object or a three-dimensional display object. The result is poor operability.
(2) An object handled by a conventional computer system has only planar coordinates and no depth coordinates to define the location at which the object is displayed on the display. Consequently, disposition in the depth direction in accordance with user preference and operating environment in the depth direction cannot be selected.
The following problems arise with the binocular parallax arrangement owing to the fact that the stereoscopic image is displayed using the binocular parallax of the left and right eyes:
(3) In a stereoscopic display using binocular parallax, convergence and focal length differ, unlike a situation in which the user observes events in real space. This type of display does not lend itself to observation over an extended period of time for physiological reasons, and long-term viewing can cause the user discomfort.
(4) With a stereoscopic display of the parallax barrier or lenticular type using binocular parallax, the area capable of being observed stereoscopically is small and it is necessary for the user to hold his or head steady in the stereoscopic viewing area or for the viewpoint of the user to be sensed so that the stereoscopic viewing area can be made to follow the user.
(5) In an apparatus configured to switch between a two-dimensional display and a three-dimensional display, the two-dimensional display portion of the display does not have an image array suited to the three-dimensional display when the three-dimensional display is being presented. The result is a display that is difficult for the user to perceive.
The arrangement which presents a three-dimensionally visible image display using volume-scanning has the following drawbacks:
(6) Image forming means capable of movement at high speed (the oscillation screen) and having a size the same as that of the three-dimensional image to be reproduced is required. This results in an apparatus of large size. In addition, achieving high-speed drive is difficult.
(7) Since an object (the image forming means for high-speed movement) having a large mass moves at high speed in image reproduction space, the apparatus can be hazardous at the time of observation.